Ship YouTube's challenge solver without costing anyone a reinstall - #446
Merged
Conversation
added 3 commits
August 25, 2026 16:26
) Extraction rides a fallback client that skips YouTube's signature/n-challenge. It works today, yt-dlp has deprecated it, and fixing that needs two things: the solver script and a JavaScript engine to run it. The obvious route was a yt-dlp-ejs dependency and a binary in data/. Both would have reached only fresh installs. runtimeId is sha256(uv.lock) and the desktop updater stands down when it changes, because it can replace backend/ but never python/. A new dependency would therefore have sent every existing desktop install to a manual reinstall. The in-app updater is something users have been told works; spending that to ship a fix nobody has asked for is a bad trade. So both halves go where the updater can reach them. The solver is vendored into app/_vendor, which becomes backend/app/_vendor in a package. 53 KB of pure Python that StemDeck never imports and yt-dlp discovers at runtime, so it is a payload rather than a dependency, and it belongs in the app layer. app/__init__ puts it on sys.path before anything else, which is the only place that can work: yt-dlp resolves its optional dependencies at import time and pipeline.download imports yt_dlp at module level. uv.lock is untouched, so runtimeId does not move. The engine goes to backend/jsruntime rather than data/jsruntime for the same reason: the updater replaces backend/ and leaves data/ alone. quickjs-ng, pinned by version and SHA256 like the macOS FFmpeg download (#172), fetched at package time by all three build scripts. 2 MB against deno's 110 MB, against a 2 GiB asset cap this project has already hit (#318, #222). Vendoring usually means remembering to update something. scripts/update_vendored_ejs.py removes that: it exits non-zero when PyPI has a newer release and takes it with --apply, verifying the wheel checksum first. Verified with nothing pip-installed: the solver resolves from app/_vendor, yt-dlp finds it, quickjs runs, and six audio formats resolve where the solverless path found one. Two things unchanged and already on the issues. The web client still fails for want of a PO token yt-dlp cannot mint, so a flagged IP may still need cookies. And the solver costs about 7.8 s per extraction, measured.
A cross-language contract with nothing checking it. Three shell and PowerShell scripts decide where the JS engine lands; Python decides where to look for it. A mismatch does not fail a build or a test, it shows up as YouTube imports quietly degrading on one platform, because yt-dlp just falls back to the client that skips the challenge. Two tests. One pins the packaged layout: in a package this file is backend/app/core/config.py, so parents[2] is backend/, and a refactor of that index would leave every desktop package without an engine. The other reads the three scripts and asserts they still install into backend/ and still verify a checksum before trusting the binary.
Swept in by a careless 'git add scripts/'. It is unrelated work in progress that was untracked before this branch started, and it belongs to whoever is writing it, not to a change about YouTube's challenge solver. The file is untouched on disk.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #438
Closes #432
Extraction currently rides a fallback client that skips YouTube's signature/n-challenge. It works today, yt-dlp has deprecated it, and fixing that needs two things: the solver script and a JavaScript engine to run it.
The updater is the constraint, not the fix
The obvious route was a
yt-dlp-ejsdependency and a binary indata/. Both would have reached only fresh installs.runtimeIdissha256(uv.lock)and the updater stands down when it changes, because it can replacebackend/but neverpython/. A dependency would therefore have sent every existing desktop install to a manual reinstall. The in-app updater is something users have been told works; spending that to ship a fix nobody has asked for yet is a bad trade.So both halves go where the updater can reach them.
app/_vendor/yt_dlp_ejs/→backend/app/_vendor/backend/jsruntime/qjsuv.lockis untouched. Verified by computing the id both sides:The solver
Vendored rather than declared. 53 KB of pure Python with no dependencies that StemDeck never imports and yt-dlp discovers at runtime, so it is a payload rather than a dependency and belongs in the app layer.
app/__init__puts it onsys.pathbefore anything else, which is the only place that can work: yt-dlp resolves its optional dependencies at import time, andpipeline.downloadimportsyt_dlpat module level.Shipping the package is also what keeps
--remote-componentsoff. yt-dlp checks for it before any remote source, so nothing is fetched and executed at runtime, which was the trust question the reporter deliberately left open in #432.The engine
quickjs-ng, fetched at package time by all three build scripts and pinned by version and SHA256, the same rule as the macOS FFmpeg download (#172).
quickjs rather than deno: 2 MB against ~110 MB, times four bundles, against a 2 GiB asset cap this project has already hit (#318, #222). deno's only remaining advantage was the PO token plugin, which needs a Node sidecar and is not shippable in a portable app.
Vendoring without the manual upkeep
scripts/update_vendored_ejs.pyexits non-zero when PyPI has a newer release and takes it with--apply, verifying the wheel checksum first.Every distribution checked
Copy-Tree app/backend/jsruntimecp -R app/backend/jsruntimecp -R app/backend/jsruntime, both archesThe wheel was built and inspected: all seven vendored entries ship, including the
.jsfiles. Architectures line up, Linux portable is x86_64-only and Docker publisheslinux/amd64only.Verified
With nothing pip-installed, to prove the vendored copy is what works:
The #432 failure no longer reproduces. Before:
n challenge solving failed→Only images are available. After:[jsc:quickjs] Solving JS challenges.A full import through
download()completes.Two tests worth naming
The build scripts and the runtime lookup are a contract across two languages with nothing checking it. A mismatch does not fail a build, it shows up as imports quietly degrading on one platform. One test pins the packaged layout (
parents[2]isbackend/), another reads all three scripts and asserts they still install intobackend/and still verify a checksum.Known, and already on the issues
The web client still fails for want of a PO token yt-dlp cannot mint, so a flagged IP may still need the cookies option from #439. #432 may not fully close on this alone.
It costs about 7.8 s per extraction, measured and repeated: 2.5 s without a runtime, 10.3 s with, no caching benefit. That buys resilience the fallback will stop providing.
The three build scripts are untested here. The Python side and the runtime lookup are verified live; the fetch-and-verify blocks ship on review and want a first real build per platform.